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© A method of reducing block artifacts created by block transform 



compression algorithms. 



© A method of improving image quality when using 
block transform image compression algorithms by 
applying a variable lowpass filter (blur) operation on 
block boundaries that is based on the coefficients of 
the transformed data. The method of reducing block 



artifacts results from adaptively blurring the block 
boundaries based on the frequency content of the 
blocks. Low frequency blocks are heavily blurred,, 
while high frequency blocks should have very little 
blur. 
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Technical Field 



This invention relates generally to a method of 
processing digital images, and in particular, to a 
method of correcting block artifacts in images 
caused by transform-based image compression. 

Background Art 



Digital images are composed of an enormous 
amount of data. Storage of this type of data on 
digital media is generally expensive and transmis- 
sion of digital images requires either a large band- 
width or a long period of time. Many algorithms 
have been developed to compress image data by 
removing redundant information from the image. 
The goal of these algorithms is to reduce the 
amount of data needed to represent the image 
while minimizing the amount of image degradation. 

One well known compression technique is 
transform coding. This method involves taking a 
transformation of the image data to provide a se- 
quence of coefficients which can be encoded us- 
ing, for example, a non-equal number of bits for 
each resulting coefficient. In particular, the number 
of bits employed is based upon the logarithm of 
the variance for a particular coefficient. At the re- 
ceiver, the coded coefficient data is employed for 
reconstructing the coefficient values and perform- 
ing the inverse, of the original transform to obtain 
an image representative of the original data. 

One form of transform coding, block image 
coding is often used to accommodate localized 
variations in image characteristics. With block im- 
age coding, a digitized image is decomposed into 
small rectangular regions (or "blocks") which are 
transform coded and, for example, transmitted 
through a digital communications channel. At the 
receiver, the blocks are decoded and re-assembled 
in order to reconstruct the image. In a typical 
situation, an image composed of an array of 
512x512 picture elements (pixels) can be viewed 
as an array of 64x64 blocks, where each block 
contains 8x8 pixels. 

Several kinds of transformations are commonly 
used for this type of coding. Typical transforms 
include the Fourier transform, cosine transform, 
Hadamard transform, and Harr transform. These 
transformations operate on an M x N block of 
image data and produce a M x N array of coeffi- 
cients. These coefficients have the property that 
they are related to specific spatial frequencies in 
the original image. Normally, the two dimensional 
array of coefficients is arranged into a one dimen- 
sional array that approximately orders the coeffi- 
cients from lowest frequency to highest frequency. 
This one dimensional array is then encoded and 
transmitted. 



Of the possible transformations, the Discrete 
Cosine Transform (DCT) is the most commonly 
used. It has been proposed as a standard for lossy 
multibit image compression by the Joint Photo- 

5 graphic Experts Group (JPEG) of the International 
Standards Organization (ISO). The DCT is popular 
because it tends to concentrate most of the in- 
formation in the original image into a smaller group 
of low-frequency coefficients in the transformed 

10 image. These coefficients can then be efficiently 
encoded to provide the required compression. 

The major disadvantage of block image coding 
is that the image is degraded by the coding pro- 
cess, and the boundaries of the reconstructed 

75 blocks can be clearly visible in the resulting image. 
In particular, this occurs because the quantization 
noise is generally correlated within blocks but is 
independent between blocks, yielding mismatches 
at block boundaries. Because of these blocking 

20 artifacts, reconstructed images appear to be com- 
posed of "tiles". 

Several techniques have been described in the 
prior art for reducing the block artifacts. Most of 
these techniques involve modifying the encoder in 

25 some way so that when the image is decoded and 
reconstructed, there are no artifacts. One approach 
is to overlap the blocks slightly, by one pixel for 
example, and reconstruct the overlapping regions 
at the receiver by using the average of the re- 

30 constructed pixels from each of the overlapping 
blocks. 

Another approach is described in US Patent 
No. 4,754,492. In this technique, the image data in 
a block is prefiltered with image data in the neigh- 

35 boring blocks, then transformed and encoded. De- 
compression consists of performing the inverse 
transform and postfiltering the blocks. 

These techniques are useful if the source and 
destination of the images are both capable of per- 

40 forming the appropriate algorithms. However, in 
many applications, a user who receives com- 
pressed images does not have control over the 
particular technique used to encode the image. For 
example, the DCT compression algorithm proposed 

45 as a standard by the JPEG committee does not 
employ any special encoding techniques to remove 
block artifacts from the decoded images. In this 
case, some other method must be used to remove 
the block artifacts. 

50 Another common technique is to use a low- 

pass filter to blur the block boundaries in the im- 
age. This technique has the advantage of not re- 
quiring special processing during the compression 
of the image. Although this technique is very effec- 

55 tive in reducing the blocking effects, it blurs high 
frequency details along the block boundaries with a 
perceptible loss of sharpness. Another way of look- 
ing at that problem is by examining the RMS error 
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in the image. In low-frequency areas, blurring the 
image tends to reduce the RMS error in the image; 
however, in high frequency areas, where consecu- 
tive pixels have less correlation, blurring the image 
substantially raises the RMS error. 

The object of the present invention is to. pro- 
vide an improved method for removing block arti- 
facts from images that have been compressed by a 
block transform compression algorithm. It is a fur- 
ther object of this invention to provide the block 
artifact removal as part of the decoding process 
only. 

Disclosure of the Inventi on 

This invention is a method of adaptively pro- 
cessing the boundaries of an image to reduce the 
block artifacts in the image without blurring high 
frequency detail within the image. The invention 
operates as a part of the compression decoder 
only. 

A block transform decoder includes a means of 
decoding the transform coefficients from the com- 
pressed image data. These coefficients are then 
inverse transformed to reconstruct an image block 
that is representative of the original data. In this 
invention, the decoded transform coefficients are 
evaluated by a frequency analyzer. The frequency 
analyzer then decides on an appropriate amount of 
blur to be applied to the boundaries of adjacent 
blocks. This decision is sent to a variable strength 
lowpass filter that blurs the pixels along the bound- 
ary between the blocks. 

This invention applies to transformations in 
which the transformed image is representative of 
the frequency content of the image. The Fourier 
transform, cosine transform, Hadamard transform, 
and Harr transform all fall into this category. Using 
this feature, the frequency analyzer can easily de- 
termine the amount of high frequency detail in the 
image by simply examining the position and mag- 
nitude of the coefficients in the transformed data. 

Brief Description of the Drawing s 

Figure 1 illustrates the theoretical transform 
model known in the art using a functional block 
diagram; 

Figure 2 illustrates the use of transforms in 
block image compression as was commonly 
known in the art using a functional block dia- 
gram; 

Figure 3 illustrates a generalized block transform 
decompression known in the art using a func- 
tional block diagram; 

Figure 4 illustrates in a functional block diagram 
of the preferred embodiment of this invention; 
Figure 5 illustrates a method of arranging 2- 
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dimensional transform coefficients into a 1 -di- 
mensional list- 
Figure 6 illustrates in a functional block diagram 
the configuration of the frequency analyzer 
5 shown in Figure 4; 

Figure 7 illustrates a functional block diagram of 
the configuration of the blur filter shown in Fig- 
ure 4; 

Figure 8 shows the locations of the pixels that 
io form a vertical boundary between two horizon- 
tally adjacent blocks; and 

Figures 9A-B illustrate a flow chart of the pre- 
ferred embodiment of the present invention. 

75 Modes of Carrying Out the Invention 

In the preferred embodiment, the invention ap- 
plies to the DCT algorithm which has been adopted 
by the JPEG ISO committee. The present embodi- 
20 ment provides a method of correcting block arti- 
facts of the DCT algorithm by adaptively blurring 
the block boundaries based on the frequency con- 
tent of the blocks. Low frequency blocks are heav- 
ily blurred, while high frequency blocks receive 
25 very little blur. 

Because the DCT transforms the image into 
the frequency domain one can adaptively blur the 
block boundaries based on the coefficients of the 
DCT; for example, the amount of blur on the fre- 
30 quency of the highest non-zero coefficient in the 
transformed blocks. 

Figure 1 illustrates the well known method of 
how to apply a transform to image data. The origi- 
nal M x N image data in block 10 can be trans- 
35 formed using any of the previously described 
transform algorithms into an array of M x N coeffi- 
cients in block 12. The newly transformed image 
has no resemblance to the original image, the 
inverse transform operation can then be applied to 
40 the transform coefficients and an image is recon- 
structed in block 14 that is representative of the 
original image. If the transform coefficients are 
stored with infinite precision, the newly reconstruct- 
ed image would be an exact duplicate of the origi- 
ns nal image. 

Figure 2 illustrates a functional block diagram 
that demonstrates how transforms are commonly 
used in image compression. First the image data is 
tiled into a set of N x N blocks in block 16 to 
so reduce the computational demands of the trans- 
formation. Each block is then transformed in block 
18, and the transform coefficients are encoded with 
some loss of precision into the compressed image 
data in block 20. The compressed data may then 
55 be stored or transmitted. To reconstruct the image, 
the compressed data is decoded to N x N blocks 
of transform coefficients in block 22, and inverse 
transform is applied thereto. The reconstructed im- 
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age blocks in block 24 are then reassembled into a 
new image that is representative of the original 
image. 

Figure 3 shows a typical prior art block trans- 
form decoder. The decoder 26 reads the com- 
pressed input image data and creates a block of 
decoded transform coefficients. The inverse trans- 
form 28 takes the coefficients outputted by de- 
coder 26 and performs the inverse transform op- 
eration to create a reconstructed block of the im- 
age image. 

The preferred embodiment of the present in- 
vention is shown in Figure 4. The system shown in 
Figure 3 is in box 30 and has added thereto a 
frequency analyzer 32 and a filter 34. Compressed 
image data enters decoder 26 which generates a 
block of decoded transform coefficients, T^ (for the 
block in the ith column and the jth row of the tiled 
image). These coefficients are then sent to inverse 
transform 28 and frequency analyzer 32. Inverse 
transform 28 transforms the decoded coefficients 
into a block of reconstructed image data, Rjj repre- 
sentative of the original image but with the block 
artifacts. The image data from decoder 26 is also 
passed to frequency analyzer 32 which analyzes 
the magnitude of the coefficients in block T^ so as 
to determine the amount of blur required to remove 
the block artifacts between adjacent blocks. A verti- 
cal blur factor is generated to control the 
amount of blur applied to the vertical boundary 
between the current reconstructed image block, Ry 
and the previous block on the current line, j. A 
horizontal blur factor is generated to control the 
amount of blur applied to the horizontal boundary 
between the current reconstructed image block, Ry, 
and the adjacent block on the previous line, Rj p. 
These decisions are sent to filter 34. Filter 34 blurs 
the pixels along the block boundaries with adjacent 
pixels from neighboring blocks to generate a new 
image representation of the original but without 
block artifacts. 

The technique of ordering the coefficients in a 
zigzag pattern shown in Figure 5 is well known. 
Use of this technique corresponds roughly to or- 
dering the coefficients in order from those repre- 
senting the lowest spatial frequencies to those re- 
presenting the highest spatial frequencies. This 
pattern is used because it groups similar frequen- 
cies together to allow for more efficient encoding of 
the data. 

In accordance with our preferred embodiment, 
the amount of blur along block boundaries is based 
on the highest non-zero coefficient in the adjacent 
blocks. The 8x8 array of transformed values are 
placed in a 1 -dimensional list in approximately in- 
creasing frequency. Figure 5 illustrates how the 8 x 
8 array of coefficients are transformed into a 1- 
dimensional list in approximately increasing fre- 
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quency. Because the value of the highest non-zero 
coefficient or cutoff frequency in this list is known 
as a result of the decoding procedure, this value 
may be stored for later use in the frequency ana- 
lyzer (32). 

The function of the frequency analyzer 32 is to 
examine the one dimensional list for each block 
and determine the appropriate amount of blur to 
apply to the block boundaries. In the preferred 
embodiment shown in Figure 6, the activity level 
Aj. is determined by the activity analyzer 36. The 
activity level corresponds to the position of the last 
coefficient in the list whose value is above a pre- 
determined threshold. The activity level Ay, is se- 
lected in this way for efficiency. The activity level is 
then stored in storage block 38 while the activity 
levels for the two adjacent block, A M j and Aj j., are 
retrieved from storage. These three values are 
passed to the blur selection block 33 to determine 
the required amount of blur, B v and B h , to apply to 
the horizontal and vertical boundaries between the 
current reconstructed image block R^ and the pre- 
vious adjacent reconstructed image blocks. 

The threshold is normally set to zero which 
results in a very efficient implementation. The de- 
coder block 26 identifies the location of the last 
non-zero coefficient in the encoded list because 
the standard JPEG algorithm requires that an End- 
of-BIock code by transmitted after the last non-zero 
coefficient. 

In the preferred embodiment, the two blur fac- 
tors, B v and B h , are determined within the blur 
selection block 33 by using the activity levels from 
the two adjacent blocks (A^ and Am j for B VJ and A^ 
and Aj H for B h ) as indices into a two-dimensional 
table of blur values. The blur values in the table are 
experimentally determined to minimize the RMS 
error in the pixel along the boundaries. As a result, 
the block artifacts are removed while reducing the 
RMS error in the reconstructed image. 

The blur values are numbers between zero and 
one that are used by blur filter 34. For each pixel 
along the boundary the filter uses the original value 
of the pixel Pi , and the value of the adjacent pixel 
in the neighboring block P 2 , to determine a new 
value Pr that replaces the original value. P/ is 
calculated by the following equation: 

Pr = (Pt x B) + (P 2 x (1-B)) 

Figure 7 shows the filter design that executes 
the steps set forth by the above equation. This 
filter design provides good artifact removal and can 
be implemented efficiently. Larger, more complex 
filters could be used to provide more thorough 
artifact removal resulting in unfortunately, larger 
execution time. 
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Flow Chart 

Figures 9A-B shows a flow chart for the inven- 
tion. Block 100 initializes variables i and j to keep 
track of the row and column of the image block 5 
being decoded. Block 101 reads enough com- 
pressed data to decode the transform coefficients 
for the current image block. Block 102 decodes the 
compressed data to create an NxN block of trans- 
form coefficients, T M . Block 103 performs the in- 10 
verse transform on T (j to create an NxN block of 
reconstructed image data R,j. Block 104 analyzes 
the block of transform coefficients T tj , and assigns 
an activity level A,j to the image block. Decision 
block 105 checks to see if the current image block 15 
is the first block on the line to determine if there is 
a vertical boundary with the previous reconstructed 
image block R M j. Block 106 selects the amount of 
blur B v , that should be applied to the vertical 
boundary between the current block and the pre- 20 
vious block based on the activity levels in the 
blocks Ay and A M j. Block 107 performs the blur on 
the vertical block boundary. Decision block 108 
checks to see if the current block is in the first line 
of the image to determine if there is a block above 
that forms a horizontal boundary with the current 
image block. Block 109 selects the amount of blur 
B h> that should be applied to the horizontal bound- 
ary between the current block Rj j( and the cor- 
responding block on the previous line R f f1 , based 
on the activity levels in the blocks, Ay and Aj j. 1t 
block 110 performs the blur operation on the hori- 
zontal boundary. Block 111 increments the column 
counter, i, and decision block 112 tests to see if 
the end of the current image line has been 
reached. If not, the next block on the line is pro- 
cessed in the same manner. When the end of the 
image line is reached, the column counter is reset 
to 0 and the line counter is incremented in block 
1 1 3. Decision block 1 1 4 checks to see if the end of 
the image has been reached. If not, the next line is 
processed in the same manner, otherwise the pro- 
cess is finished. 

It should be understood that more complicated 
algorithms can be used for the frequency analyzer, 
for example, by also examining the amplitude of 
the low frequency coefficients or alternatively, one 
could examine the highest horizontal and vertical 
frequencies and process horizontal and vertical 
borders differently. 

Advantages and Industrial Applica bi lity 

An important advantage of the present inven- 
tion is that the image quality of the images that 55 
have been processed by JPEG DCT is improved 
without modifying the compression algorithm that is 
being considered as a standard. It should be un- 
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derstood that while the filtering concept of the 
present invention has been described in terms of 
the JPEG DCT standard, it is equally applicable to 
any DCT based compression algorithm, and more 
generally to any frequency transform based al- 
gorithm. 

Another way of practicing the present invention 
is to modify the software implementation of the 
DCT algorithm to provide the filtering operation of 
the present invention. The filter operation is very 
fast and does not degrade the image in high fre- 
quency areas. 

The present invention will find applicability in 
the transmission and storage of digitized images 
that have been processed using a compression 
algorithm that incorporates a frequency transform. 

Claims 

1- A method for decoding block transformed im- 
ages wherein the block boundaries are adap- 
tively processed to reduce block artifacts in 
the reconstructed image comprising the steps 
of: 

decoding the transform coefficients from 
the compressed image data; 

apply the inverse transform to the coeffi- 
cients obtained from the compressed image 
data; 

frequency analyze the decoded transform 
coefficients to determine the frequency content 
of the image along the boundary between 
blocks; and 

adjust the lowpass blur filter in response to 
the analyzed frequency content of the trans- 
formed image along block boundaries. 

2. A method for decoding compressed image 
data using block transforms that subdivide 
each image into blocks of points comprising 
the steps of: 

decoding the compressed image data tp 
blocks of transform coefficients; 

applying a bidirectional transformation to 
the transform coefficients; 

transmitting the transformation coefficients 
for blur control; and 

adjusting the amount of blur in a low pass 
filter along adjacent block boundaries in re- 
sponse to the transformation coefficients, 
thereby eliminating block artifacts in the de- 
compressed image. 

3. A method for decoding compressed image 
data using block transforms that subdivide 
each image into blocks of points comprising 
the steps of: 

reading the compressed image data; 
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decoding the compressed data to create a 
block of transform coefficients; 

applying the inverse transform to said 
transform coefficients to provide a block of 
reconstructed image data; 5 

analyzing the block of transform coeffi- 
cients to determine the level of activity in said 
block; and 

adjusting the blur in a variable strength 
blur filter based upon said activity. io 

An apparatus for decoding compressed image 
data using block transformation that subdivides 
each image into blocks of points comprising: 

means for decoding the compressed im- 75 
age data into transform coefficients; 

means for applying an inverse transform to 
said transfer coefficients to provide a com- 
pressed image signal to a lowpass blur filter; 

means for analyzing the frequency of the 20 
decoded transform coefficients; 

means for adjusting the amount of blur in a 
lowpass blur filter in accordance with the fre- 
quency content so as to remove the block 
artifacts along adjacent block boundaries. 25 



10. A method for decoding images wherein bound- 
aries are adaptively processed to reduce arti- 
facts in the reconstructed image comprising 
the steps of: 

decoding the transform coefficients from 
the compressed image data; 

apply the inverse transform to the coeffi- 
cients obtained from the compressed image 
data; 

frequenzy analyze the decoded transform 
coefficients; and 

adjust filter in response to the analyzed 
frequency content. 



An apparatus for decoding compressed image 
data using block tranformation that subdivides 
each image into blocks of points comprising: 

means for reading the compressed data; 

means for decoding the compressed data 
to create a block of transform coefficients; 

means for applying the inverse transform 
to the transform coefficients resulting in a 
block of reconstructed image data; 

means for analyzing the transform coeffi- 
cients in the block to determine the activity 
level in said block; and 

means for setting said blur level of a vari- 
able strength blur filter based upon said activ- 
ity level of said block. 
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6. In the method or appartus set forth in any of 
claims 1 to 5 wherein the transform is a Dis- 
crete Cosine. 



45 



In the method or appartus set forth in any of 
claims 1 to 5 wherein the transform takes the 
form of a Fouriere transform. 

In the method or appartus set forth in any of 
claims 1 to 5 wherein the transfer takes the 
form of a Hadamard transform. 



50 



In the method or appartus set forth in any of 
claims 1 to 5 wherein the transform takes the 
form of Harr transform. 
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A method of reducing block artifacts created by block transform 



compression algorithms. 



© A method of improving image quality when using 
block transform image compression algorithms by 
applying a variable lowpass filter (blur) operation on 
block boundaries that is based on the coefficients of 
the transformed data. The method of reducing block 
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artifacts results from adaptively blurring the block 
boundaries based on the frequency content of the 
blocks. Low frequency blocks are heavily blurred, 
while high frequency blocks should have very little 
blur. 
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